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Abstract 

This letter assesses the performance on Landsat-7 images of a modified version of a cloud 
masking algorithm originally developed for clear-sky compositing of Moderate Resolution 
Imaging Spectroradiometer (MODIS) images at northern mid-latitudes. While data from recent 
Landsat missions include measurements at thermal wavelengths, and such measurements are also 
planned for the next mission, thermal tests are not included in the suggested algorithm in its 
present form to maintain greater versatility and ease of use. To evaluate the masking algorithm 
we take advantage of the availability of manual (visual) cloud masks developed at USGS for the 
collection of Landsat scenes used here. As part of our evaluation we also include the Automated 
Cloud Cover Assesment (ACCA) algorithm that includes thermal tests and is used operationally 
by the Landsat-7 mission to provide scene cloud fractions, but no cloud masks. We show that the 
suggested algorithm can perform about as well as ACCA both in terms of scene cloud fraction 
and pixel-level cloud identification. Specifically, we find that the algorithm gives an error of 
1.3% for the scene cloud fraction of 156 scenes, and a root mean square error of 7.2%, while it 
agrees with the manual mask for 93% of the pixels, figures very similar to those from ACCA 
(1.2%, 7.1%, 93.7%). 


Index Terms 

Clouds, Enhanced Thematic Mapper, Landsat, Moderate Resolution Imaging Spectroradiometer 
(MODIS), masking, Operational Land Imager (OLI), remote sensing, satellite, Thermal InffaRed 
Sensor (TIRS) 
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I. INTRODUCTION 

The presence of clouds in images acquired by the Landsat program is usually an undesirable, but 
generally unavoidable fact. With the emphasis of the program being on land imaging, suspended 
liquid/ice particles fully or partially obscure the desired observational target. Knowledge of 
cloud amount in a Landsat scene and the location of clouds is therefore valuable information that 
facilitates proper scene selection by Landsat data users, scene compositing from multiple scenes, 
and scheduling of future acquisitions [1]. Presently, Landsat-7 images come with metadata that 
provide the total cloud fraction of the scene (the fraction of cloudy over the total number of 
pixels) as well as the cloud fraction in each of the four scene quadrants. These cloud “scores” are 
generated by the Automated Cloud Cover Assessment (ACCA) algorithm [2], Unfortunately, a 
classification of individual pixels as either cloudy or cloud-free (i.e., a “cloud mask”) is not 
provided, forcing data users to perform their own cloud screening whenever their application 
requires it. This will change for the next Landsat mission, the Landsat Data Continuity Mission 
(LDCM), for which a cloud mask product is planned [3]. 

The purpose of this letter is to revisit a simple clear pixel detection algorithm developed for 
MODIS 250/500 m land bands [4], unassisted by thermal data, and examine whether it can 
provide pixel-level clear-cloudy sky discrimination for Landsat scenes at very small 
computational cost. While we apply the algorithm in this paper only to Enhanced Thematic 
Mapper Plus (ETM+) Landsat-7 data, it should be also applicable to historic Landsat-4 and 
Landsat-5 data from the Thematic Mapper instrument, as well as data from the Operational Land 
Imager (OLI) sensor of the upcoming LDCM. The cloud-shadow detection component, of the 
original algorithm has not yet been fully validated in our Landsat implementation and will not be 
further discussed in this paper. 
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24 II. THE CLOUD MASKING ALGORITHM 

25 The clear/cloud mask scheme introduced in [4] (hereafter “LTK scheme”) is a simple threshold 

26 scheme that uses only four MODIS 250/500 m resolution bands, specifically bands 1, 2, 3, and 6. 

27 These bands have approximate spectral equivalents in the ETM+ instrument aboard Landsat-7 

28 (Table 1). The LTK scheme threshold selection for surface type classification and cloud 

29 detection is based on typical spectral signatures of five major pixel classes: non-vegetated land, 

30 vegetated land, water, ice/snow and cloudy pixels, as depicted in Fig. 6 of [4], The scheme 

31 successively applies threshold tests to first classify non-vegetated pixels, followed by the 

32 classification of ice/snow, water, and cloudy pixels. Any pixels not classified to any of the above 

33 classes are assigned to the vegetated class. A flow chart presenting our modified LTK scheme is 

34 provided in Fig. 1. After exhaustive testing of a variety of plausible adjustments to the LTK 

35 thresholds to improve its performance, we settled on two threshold modifications in the last step 

36 of the algorithm that separates cloud and vegetated pixel classes. These new thresholds resulted 

37 in substantially better agreement between the cloud/clear masks from LTK and those from a 

38 manual “truth” mask (discussed below) for a large collection of ETM+ scenes. Both the original 

39 and modified LTK scheme threshold values are provided in the last box of the Fig. 1 flow chart, 

40 and the performance of both variants of the scheme are contrasted in the next section. The 

41 decrease of the ETM+ band 1 threshold (MODIS band 3) is consistent with the values of the 

42 spectral reflectance plot for vegetated land shown in Fig. 6 of [4], which do not seem to exceed 

43 0.1. However, the decrease of the band 5 threshold seems somewhat inconsistent with the 

44 observed values of MODIS band 6 reflectances in the same plot, which seem to range between 

45 0.1 and 0.18. The fact that a lower value appears to work better for Landsat may be due to the 

46 difference in spectral range and central wavelength location of the MODIS and Landsat bands. 
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47 Numerous other threshold modifications also improved upon the original scheme, but none 

48 worked as well as the two modifications that were eventually adopted. While we realize that a 

49 cloud masking scheme developed for an instrument with similar spectral characteristics, but with 

50 bands of different spectral widths, different spatial resolution, and off-nadir viewing capabilities, 

51 should not necessarily translate perfectly to ETM+, we found nevertheless that in practice the 

52 LTK scheme carries over quite well from MODIS to Landsat observations. 

53 

54 m. ALGORITHM IMPLEMENTATION ON LANDSAT-7 SCENES 

55 The modified LTK scheme is applied to a collection of 156 Landsat scenes, a subset of the 212 

56 scenes used by [2] to evaluate the performance of ACCA scene-averaged cloud fractions. The 

57 criteria used to select the original dataset of 212 scenes is provided in [2], These scenes are 

58 approximately evenly apportioned among 9 latitude zones covering the entire globe. The present 

59 subset of 156 consists of the scenes for which it was determined by USGS-EROS personnel that 

60 a reliable cloud mask can be obtained. The manual mask was developed via a visual assessment 

61 procedure [5]: Three experienced USGS imagery analysts performed manual assessment of the 

62 scenes in [2], 11 scenes were examined by all three in order to obtain the approximate error of 

63 the procedure, which was found to be about 7% on average [5]. The process involved opening 

64 each full resolution scene in Adobe Photoshop in a variety of RGB combinations, including 

65 overlays of the (resampled) thermal band when necessary. The analysts then used appropriate 

66 Photoshop image processing functions to isolate clouds. Two classes of clouds were identified: 

67 thick and thin. Cloud pixels were labeled as thin if they were transparent but still visually 

68 identifiable as clouds. For the purposes of this paper, no distinction is made between thin and 

69 thick clouds in the quantitative metrics of the LTK scheme performance, but only when 
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70 interpreting the results. Further quality evaluation of the visual mask was performed by an expert 

71 remote sensing group at Boston University (BU) [6], which recommended that the collection of 

72 156 scenes be further reduced by 14. Here we provide results for both the 156 and 142 scene 

73 sets. As will be shown, both the ACCA and the LTK schemes agree better with the manual mask 

74 for the smaller subset, a fact that seems to give further credence to the BU evaluation. In addition 

75 to the USGS manual mask, we also have for these scenes the manually-determined cloud fraction 

76 used as “truth” in [2], The set of 156 contains 35 scenes from midlatitudes (30°-45°N or 30°- 

77 45°S), 41 scenes from the subtropics (15°-30°N or 15°-30°S), 21 scenes from the tropics (15°S - 

78 15°N), 33 scenes from austral or boreal latitudes (45°-60°S or 45°-60°N), and 26 scenes from the 

79 polar regions (60°-90°N or 60°-90°S). The BU group flagged as unreliable 7 polar visual masks 

80 (4 from south and 3 from north), 1 austral mask, 2 tropical masks, 1 midlatitude south mask, and 

81 3 midlatitude north masks. The fact that the original polar group of 44 scenes was reduced by the 

82 combined manual mask screening re-evaluation of USGS and BU to 19 scenes should come as 

83 no surprise, since cloud/ice/snow discrimination is very difficult even in visual image analysis. 

84 Fig. 2 shows the outcome resulting from the original LTK scheme for a sample scene with 

85 clear vegetated and non-vegetated land pixels, water pixels, and a fair amount of cloudy pixels. 

86 The scheme appears to perform a reasonably good pixel classification and the clear/cloud mask, 

87 although slightly worse than ACCA, makes the correct distinction between clear and cloudy 

88 pixels more than 90% of the time. 

89 The overall performance of the LTK scheme in terms of the “cloud score” (the cloud 

90 fraction of the entire scene) can be seen in Fig. 3 for the 156 scene (top) and the 142 scene 

91 (bottom) sets. The left panels correspond to the original LTK scheme and the right panels to our 

92 modified version. ACCA results are included for comparison. The legends in each plot also 
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contain summary metrics such as the overall bias in scene cloud fraction, the root mean square 
error of the scene cloud fraction, and the number of “bad” scenes, defined here as scenes with 
LTK or ACCA cloud fraction absolute differences from the manual mask (“cloud fraction 
errors”) that exceed 10% (=0.1 when cloud fraction is measured in a scale from 0 to 1). 
Modifying the LTK scheme results in noticeable improvements which bring it on par, according 
to our performance metrics, with the more complex ACCA scheme which includes thermal tests. 

Using the same panel arrangement as in Fig. 3, Fig. 4 shows the performance of the 
original and modified LTK schemes in terms of the percentage of pixels for which the algorithms 
agree that a pixel is clear or cloudy (“mask agreement”). Again, the ACCA results are included 
for comparison, allowing us to create a scatterplot of this metric where each scene is represented 
by a point. Lines at the 80% agreement level are meant to isolate the poorer performers, 
discussed further below. The benefits of modifying the LTK scheme are evident, as it more 
closely approaches ACCA levels of performance. Note that that only 9 scenes have LTK mask 
agreements below 80% (4 for the set of 142), but still always above 65%. 

If good cloud masking capabilities is the objective, then the mask agreement of Fig. 4 is a 
better evaluator of the scheme’s skill. If only the scene-average cloud fraction (score) is of 
interest, and cancelling pixel misclassifications are tolerable, then the results of Fig. 3 are more 
relevant. An obvious question is whether our collection of scenes includes cases with small 
scene-average cloud fraction errors, but low mask agreement. Fig. 5 is a scatterplot of mask 
agreement vs. cloud fraction error. As expected, there is a strong anticorrelation between the two 
quantities. Scenes with small cloud fraction error usually exhibit high values of mask agreement. 
With an arbitrary choice of 5% cloud fraction error and 80% mask agreement, only one scene 
falls in the quadrant that indicates good cloud fraction estimates due to cancelling errors. 
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116 We now examine why our modified LTK algorithm performs poorly for certain scenes, 

117 either in terms of cloud fraction errors or mask agreements. We identified scenes belonging to 

118 both of these categories of poor performance based on 10% (17 out of 156 scenes) and 80% (9 

119 out of 156 scenes) thresholds, respectively, in order to investigate this question. 7 of the 9 scenes 

120 that do not pass the 80% mask agreement threshold also belong to the subset of 1 7 scenes that do 

121 not satisfy the 10% cloud fraction error criterion, so the number of unique “bad” scenes is 19. 

122 These 19 scenes have the following characteritics: 

123 (a) 5 belong to the south pole latitude zone where cloud discrimination from ice and snow is 

124 notoriously difficult. 

125 (b) 7 exhibit greater than 10% cloud fraction error also between the manual USGS cloud scores 

126 and the manual cloud scores of [2], which suggests that these scenes pose cloud identification 

127 challenges even when visually inspecting RGB composites. 

128 (c) 7 exhibit also poor ACCA performance (greater than 10% scene cloud fraction error); only 

129 three of these belong to the 7 of category (b), yielding a total of 1 1 “difficult” scenes. 

130 (d) For these 1 1 scenes, 6 have less than 80% pixel-level agreement for the ACCA algorithm as 

131 well, and 7 were deemed to have unreliable USGS visual masks by the BU team (i.e. they 

132 belong to the set of 1 56, but not to the set of 142). 

133 (e) 12 of the 19 scenes have high amounts of thin clouds, specifically a ratio of thin cloud pixels 

134 to total number of cloudy pixels higher than the median value of 0.31 (derived from the 134 

135 out of 156 scenes with non-zero cloudiness). 3 scenes have actually a ratio greater than 0.9 

136 while only 2 have a ratio smaller than 0.1. Thin cloud is very difficult to identify in land- 

137 dominated scenes with a simple threshold algorithm relying only on solar bands. The LTK 

138 scheme should therefore be used with caution for cloud masking when visual image 
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inspection or other evidence (e.g., thermal band signatures) indicates the presence of thin 
clouds. 

IV. SUMMARY AND CONCLUSIONS 

We have revisited a cloud/clear masking algorithm initially developed for MODIS clear-image 
compositing and applied two threshold modifications that significantly improve its performance 
when applied to a set of 156 Landsat scenes selected to cover the full range of Earth 
geographical zones. The algorithm uses four Landsat solar bands that roughly correspond to the 
MODIS bands of the original algorithm. We have found that despite its simplicity the algorithm 
works quite well, giving a bias error of 1 .3% for the scene cloud fraction of the 1 56 scenes, and a 
root mean square error of 7.2%. The algorithm agrees with the pixel classification (clear/cloudy) 
of a manual (visual) mask for 93% of the pixels, on average. These performance metrics (1.3%, 
7.2%, 93.0%) are very close to those (1.2%, 7.1%, 93.7%) of the more sophisticated Landsat-7 
operation algorithm (ACC A), which also incorporates thermal band tests. 

Two motivations for bypassing thermal tests are simplicity and speed. The modified LTK 
scheme of this paper can be coded much easier by a non-expert than ACCA with its involved 
“pass two” portion which, while helpful for re-classifying ambiguous pixels, has the drawback of 
added complexity and greater execution time. Another reason to consider the scheme of this 
paper, that may become relevant for future missions such as LDCM, is that cloud masking can 
continue to operate even if no thermal data are available. This possibility is certainly not remote 
for LDCM given the fact that solar and thermal sensing capabilities will be partitioned between 
two instruments, OLI and TIRS, the latter of which has a shorter design life. While this modified 
LTK scheme can be applied to historical Landsat data, its availability for future Landsat 
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acquisitions is also important since it provides an extra cloud masking assessment opportunity 
for whatever operational cloud mask algorithm is eventually adopted. The algorithm can also be 
part of consensus cloud masks or masks with confidence level flags based on the degree of 
agreement between an ensemble of distinct masking schemes. 

Weaknesses of the modified LTK scheme that we have exposed in this work include its 
limited ability to identify thin clouds and clouds over snowy or icy surfaces. Further work can 
conceivably be undertaken to add a thermal component to the LTK algorithm and/or to add 
threshold tests for the 1.38 pm “thin cirrus” band of LDCM’s OLI instrument. Both of these 
elements have the potential to improve the LTK-based scheme significantly. Finally, it is 
recognized that the testing of a masking algorithm on a collection of 156 scenes is not exhaustive 
or conclusive, even if the scenes were selected to encompass most of the surface, solar geometry 
and cloud type diversity encountered around the globe. Unfortunately, scheme evaluations that 
involve manually generated masks cannot by nature be very extensive because the laborious 
nature of visual pixel classification. 
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199 List of Tables 

200 Table I 

201 Correspondence Between MODIS and ETM+ Bands 


MODIS 

ETM+ 

Band 1 (620-670 ran) 

Band 3 (630-690 ran) 

Band 2 (841 -876 ran) 

Band 4 (750-900 ran) 

Band 3 (459-479 ran) 

Band 1 (450-515 ran) 

Band 6 (1628-1652 ran) 

Band 5 (1550-1750 nm) 


202 

203 Correspondence between the MODIS bands used in the original LTK scheme and the ETM+ 

204 bands used in this study for the modified LTK scheme. 

205 

206 List of Figures 

207 Figure 1. Flow chart depicting the LTK clear/cloud masking part of the LTK scheme. The 

208 numbers in parentheses in red are the original LTK thresholds for the reflectances of the 

209 equivalent MODIS bands. 

210 Figure 2. An example of applying the original LTK scheme on a Landsat-7 scene acquired on 

211 April 22, 2001 . (top) true color RGB image; (bottom) LTK pixel classification. 

212 Figure 3. Comparison between manually-determined and cloud mask algorithm scene cloud 

213 fractions (ACC A or LTK). Top row corresponds to the set of 156 Landsat scenes, and bottom 

214 row to the reduced set of 142 scenes (see text for details). The left plots are for the original LTK 

215 scheme, while the right plots show results after our modification. “Bad” in the legends refers to a 

216 scene for which the cloud fraction error is greater than 10% (=0.1 when cloud fraction is 

217 measured in a scale from 0 to 1 ). 

218 Figure 4. Comparison between LTK and ACCA mask agreement (in %) for the original LTK 

219 scheme (left panels) and the modified LTK scheme (right panels). The top row is for the set of 

in 
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226 


156 Landsat scenes while the bottom row is for the reduced set of 142 Landsat-7 scenes. “Bad” 
in the legends refers to a scene for which the mask agreement is less than 80%. 

Figure 5. Scatterplot of the mask agreement of the modified LTK scheme against cloud fraction 
error. The left lower quadrant identifies the number of scenes (one in this case) where a low 
cloud fraction error (< 5%) can be achieved by cancellation of pixel misidentifications (as 
measured by the % mask agreement-less than 80% is considered poor performance). 
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Figure 1 . Flow chart depicting the LTK clear/cloud masking part of the LTK scheme. The 
numbers in parentheses in red are the original LTK thresholds for the reflectances of the 
equivalent MODIS bands. 
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Figure 2. An example of applying the original LTK scheme on a Landsat-7 scene acquired on 
April 22, 2001 . (top) true color RGB image; (bottom) LTK pixel classification. 
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Figure 3. Comparison between manually-determined and cloud mask algorithm scene cloud 
fractions (ACC A or LTK). Top row corresponds to the set of 156 Landsat scenes, and bottom 
row to the reduced set of 142 scenes (see text for details). The left plots are for the original LTK 
scheme, while the right plots show results after our modification. “Bad” in the legends refers to a 
scene for which the cloud fraction error is greater than 10% (=0.1 when cloud fraction is 
measured in a scale from 0 to 1). 
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Figure 4. Comparison between LTK and ACCA mask agreement (in %) for the original LTK 


scheme (left panels) and the modified LTK scheme (right panels). The top row is for the set of 


156 Landsat scenes while the bottom row is for the reduced set of 142 Landsat-7 scenes. “Bad” 


in the legends refers to a scene for which the mask agreement is less than 80%. 
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Figure 5. Scatterplot of mask agreement of the modified LTK scheme against cloud fraction 
error. The left lower quadrant identifies the number of scenes (one in this case) where a low 
cloud fraction error (< 5%) can be achieved by cancellation of pixel misidentifications (as 
measured by the % mask agreement-less than 80% is considered poor performance). 
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